hrw4u: Sync Python transpiler and add roundtrip tests#13058
Open
zwoop wants to merge 1 commit intoapache:masterfrom
Open
hrw4u: Sync Python transpiler and add roundtrip tests#13058zwoop wants to merge 1 commit intoapache:masterfrom
zwoop wants to merge 1 commit intoapache:masterfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Synchronizes the Python HRW4U transpiler / inverse transpiler with recent header_rewrite behavior, normalizes autest rule corpora for stable bidirectional conversion, and adds tests to verify hrw4u↔conf roundtrips against the gold_tests rules.
Changes:
- Add pytest-based validation that hrw4u output matches the autest
.confrule files, plus reverse conversion tests. - Extend hrw4u/u4wrh symbol tables and inverse visitor logic (including new ops and section/condition handling) and expose
--no-merge-sectionsinu4wrh. - Normalize and add autest HRW4U source files and update corresponding
.conffiles for consistent quoting/indentation/modifiers.
Reviewed changes
Copilot reviewed 61 out of 62 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/hrw4u/tests/test_autest_rules.py | New forward (hrw4u→conf) tests against autest gold rules. |
| tools/hrw4u/tests/test_autest_rules_reverse.py | New reverse (conf→hrw4u) tests against autest gold rules. |
| tools/hrw4u/tests/data/ops/skip-remap.input.txt | Update skip-remap test to PRE_REMAP section. |
| tools/hrw4u/tests/data/ops/skip-remap.output.txt | Update expected hook mapping for PRE_REMAP. |
| tools/hrw4u/tests/data/ops/skip-remap.ast.txt | Update AST expected for PRE_REMAP section name. |
| tools/hrw4u/tests/data/ops/set-effective-address.input.txt | Add forward test input for set-effective-address. |
| tools/hrw4u/tests/data/ops/set-effective-address.output.txt | Add expected conf output for set-effective-address. |
| tools/hrw4u/tests/data/ops/set-effective-address.ast.txt | Add expected AST for set-effective-address. |
| tools/hrw4u/tests/data/ops/set-cc-alg.input.txt | Add forward test input for set-cc-alg. |
| tools/hrw4u/tests/data/ops/set-cc-alg.output.txt | Add expected conf output for set-cc-alg. |
| tools/hrw4u/tests/data/ops/set-cc-alg.ast.txt | Add expected AST for set-cc-alg. |
| tools/hrw4u/src/types.py | Add magic string identifiers for new operators. |
| tools/hrw4u/src/tables.py | Update symbol/section restrictions and add new statement function mappings. |
| tools/hrw4u/src/hrw_visitor.py | Adjust inverse conversion behavior (section merging, if-chain handling, [L] handling, preamble insertion). |
| tools/hrw4u/src/hrw_symbols.py | Improve set parsing for quoted commas and remove unused logic. |
| tools/hrw4u/scripts/u4wrh | Add --no-merge-sections CLI flag plumbing. |
| tools/hrw4u/README.md | Document C++ native parser build requirements and setup notes. |
| tools/hrw4u/pyproject.toml | Add autest pytest marker. |
| tests/gold_tests/pluginTest/header_rewrite/rules/set_redirect.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/set_redirect.conf | Normalize conf formatting/quoting for roundtrip stability. |
| tests/gold_tests/pluginTest/header_rewrite/rules/run_plugin.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/run_plugin.conf | Normalize conf formatting/quoting for roundtrip stability. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule.conf | Normalize conf indentation for if/elif chain. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_set_header_after_ssn_txn_count.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_set_header_after_ssn_txn_count.conf | Normalize conf quoting/indentation. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_set_body_status.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_set_body_status.conf | Normalize hook lines and quoting. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_set_body_from_remap.hrw4u | Add HRW4U source version of autest rule (multiple same hook sections). |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_set_body_from_remap.conf | Normalize hook lines, quoting, and condition formatting. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_set_body_from_plugin.hrw4u | Add HRW4U source version of autest rule (multiple same hook sections). |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_set_body_from_plugin.conf | Normalize hook lines, quoting, and condition formatting. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_session_vars.hrw4u | Add HRW4U source version of autest rule (SESSION_VARS). |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_session_vars.conf | Normalize hook lines and indentation. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_server_conditions.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_server_conditions.conf | Normalize hook lines and indentation. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_procedures.hrw4u | Add HRW4U source version of autest rule (procedure usage). |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_l_value.hrw4u | Add HRW4U source version of autest rule (break/[L] semantics). |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_l_value.conf | Normalize [L] representation and indentation. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_empty_body.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_empty_body.conf | Normalize hook lines and quoting. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_effective_address.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_effective_address.conf | Normalize hook lines and quoting. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_cond_method.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_cond_method.conf | Normalize hook lines and remove [L] in favor of hrw4u control-flow. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_client.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_client.conf | Normalize hook lines/modifier ordering/indentation. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_add_cache_result_header.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/rule_add_cache_result_header.conf | Normalize hook lines and quoting. |
| tests/gold_tests/pluginTest/header_rewrite/rules/regex_tests.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/regex_tests.conf | Normalize hook lines and indentation. |
| tests/gold_tests/pluginTest/header_rewrite/rules/query_sub_key.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/query_sub_key.conf | Normalize hook lines and indentation. |
| tests/gold_tests/pluginTest/header_rewrite/rules/procs/test/stamp.hrw4u | Add procedure definition used by procedure autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/nested_ifs.hrw4u | Add HRW4U source version of nested-if autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/nested_ifs.conf | Normalize nested if/elif/endif indentation and spacing. |
| tests/gold_tests/pluginTest/header_rewrite/rules/implicit_hook.hrw4u | Add HRW4U source version of implicit-hook autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/implicit_hook.conf | Normalize hook lines and indentation. |
| tests/gold_tests/pluginTest/header_rewrite/rules/glob_set_redirect.hrw4u | Add HRW4U source version of autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/glob_set_redirect.conf | Normalize hook lines and quoting. |
| tests/gold_tests/pluginTest/header_rewrite/rules/complex_logics.hrw4u | Add HRW4U source version of complex logic/grouping autest rule. |
| tests/gold_tests/pluginTest/header_rewrite/rules/complex_logics.conf | Normalize hook lines/modifiers/indentation for complex grouping cases. |
Contributor
Author
|
Address Copilot's review. |
Extract hrw4u-specific changes from apache#12825. Fixes issues in the Python transpiler found during C++ parser development, normalizes autest .conf rules for bidirectional conversion, and adds roundtrip tests verifying hrw4u<->conf equivalence.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extract hrw4u-specific changes from #12825. Fixes issues in the Python transpiler found during C++ parser development, normalizes autest .conf rules for bidirectional conversion, and adds roundtrip tests verifying hrw4u<->conf equivalence.